通过学习曲线检测过拟合和欠拟合

对于这道测验，我们将使用三个模型来训练下面的圆形数据集。

决策树模型，
逻辑回归模型，以及
支持向量机模型。

其中一个模型会过拟合，一个欠拟合，还有一个正常。首先，我们将编写代码为每个模型绘制学习曲线，最后我们将查看这些学习曲线，判断每个模型对应哪个曲线。

首先，请记住三个模型的学习曲线外观如下所示：

对于这道测验的第一部分，你只需取消注释其中一个分类器，并点击'测试答案'以查看学习曲线的图表。但是如果你喜欢编程的话，以下是一些编程详情。我们将使用函数 learning_curve ：

train_sizes, train_scores, test_scores = learning_curve(
    estimator, X, y, cv=None, n_jobs=1, train_sizes=np.linspace(.1, 1.0, num_trainings))

不需要担心该函数的所有参数（你可以在此处了解详情），这里，我们将解释主要参数：

estimator ，是我们针对数据使用的实际分类器，例如 LogisticRegression() 或 GradientBoostingClassifier() 。
X 和 y 是我们的数据，分别表示特征和标签。
train_sizes 是用来在曲线上绘制每个点的数据大小。
train_scores 是针对每组数据进行训练后的算法训练得分。
test_scores 是针对每组数据进行训练后的算法测试得分。

两个重要的现象：

训练和测试得分是一个包含 3 个值的列表，这是因为函数使用了 3 折交叉验证。
非常重要： 可以看出，我们使用训练和测试误差来定义我们的曲线，而这个函数使用训练和测试得分来定义曲线。二者是相反的，因此误差越高，得分就越低。因此，当你看到曲线时，你需要自己在脑中将它颠倒过来，以便与上面的曲线对比。

第 1 部分：绘制学习曲线

这里，我们将对比三个模型：

逻辑回归 模型。
决策树 模型。
支持向量机 模型，具有 RBF 内核，γ 参数为 1000（稍后我们将了解它们的含义）。

取消注释每个的代码，并检查所绘制的学习曲线。如果你对绘制学习曲线用到的代码感兴趣，请查看 utils.py 标签页。

Start Quiz:

quiz.py utils.py data.csv

# Import, read, and split data
import pandas as pd
data = pd.read_csv('data.csv')
import numpy as np
X = np.array(data[['x1', 'x2']])
y = np.array(data['y'])

# Fix random seed
np.random.seed(55)

### Imports
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import GradientBoostingClassifier
from sklearn.svm import SVC

# TODO: Uncomment one of the three classifiers, and hit "Test Run"
# to see the learning curve. Use these to answer the quiz below.

### Logistic Regression
#estimator = LogisticRegression()

### Decision Tree
#estimator = GradientBoostingClassifier()

### Support Vector Machine
#estimator = SVC(kernel='rbf', gamma=1000)

from sklearn.model_selection import learning_curve

# It is good to randomize the data before drawing Learning Curves
def randomize(X, Y):
    permutation = np.random.permutation(Y.shape[0])
    X2 = X[permutation,:]
    Y2 = Y[permutation]
    return X2, Y2

X2, y2 = randomize(X, y)

def draw_learning_curves(X, y, estimator, num_trainings):
    train_sizes, train_scores, test_scores = learning_curve(
        estimator, X2, y2, cv=None, n_jobs=1, train_sizes=np.linspace(.1, 1.0, num_trainings))

    train_scores_mean = np.mean(train_scores, axis=1)
    train_scores_std = np.std(train_scores, axis=1)
    test_scores_mean = np.mean(test_scores, axis=1)
    test_scores_std = np.std(test_scores, axis=1)

    plt.grid()

    plt.title("Learning Curves")
    plt.xlabel("Training examples")
    plt.ylabel("Score")

    plt.plot(train_scores_mean, 'o-', color="g",
             label="Training score")
    plt.plot(test_scores_mean, 'o-', color="y",
             label="Cross-validation score")


    plt.legend(loc="best")

    plt.show()

x1,x2,y
0.336493583877,-0.985950993354,-1.0
-0.0110425297266,-0.10552856162,1.0
0.238159509297,-0.61741666482,1.0
-0.366782883496,-0.713818716912,1.0
1.22192307438,-1.03939898614,-1.0
-1.30456799971,0.59261847015,-1.0
-0.407809098981,-0.509110509763,1.0
0.893188941965,1.18285985648,-1.0
-0.00546337259365,-0.589551228864,1.0
0.406423768278,0.611062234636,1.0
-0.145506766722,0.0365463997206,1.0
-0.0404887876421,-0.0566500319512,1.0
1.60355997627,0.0908139379574,-1.0
-0.604838450284,-0.111340204903,1.0
-0.534401237223,-1.04875779188,-1.0
0.977706756346,-1.35281793296,-1.0
-0.422036924523,-0.274418973593,1.0
1.69051344717,-0.929766839195,-1.0
0.655534595433,-0.244533046405,1.0
0.384609916121,-0.334328465856,1.0
-0.109341027267,0.273694976361,1.0
-1.28710021847,-0.406756443289,-1.0
0.435217566287,-0.192221316649,1.0
0.0555208008113,1.024011876,-1.0
1.5088217057,-0.799489053235,-1.0
0.75932306599,0.775189603256,-1.0
0.967078497167,-0.707726241999,-1.0
-0.0231301769156,1.34060202328,-1.0
-0.274591142835,-0.549682228079,1.0
-1.2080749077,-1.41385342554,-1.0
0.381259079564,-0.852947496234,1.0
0.404870623291,-0.38564643089,1.0
0.0173135930664,0.787433467901,1.0
-0.650474497449,0.377281547969,1.0
-0.175095703948,0.557524657143,1.0
0.090747012995,0.146764389396,1.0
-0.23406335446,-1.14282728744,-1.0
-0.023240502157,0.0329251073349,1.0
-0.98177853269,-0.614024199162,-1.0
0.863118366276,0.626452589641,-1.0
-0.494201528321,-1.2458627184,-1.0
0.560657440533,0.960463847964,-1.0
0.517532460272,-1.015620433,-1.0
-1.07674778462,1.64110648889,-1.0
-0.40295146753,1.74395283754,-1.0
1.26250128528,-0.0880456579187,-1.0
-1.13554604657,0.691274079866,-1.0
-1.88154070755,0.579520022541,-1.0
1.61949373896,-1.16815366758,-1.0
-0.167382068846,0.318140979545,1.0
-0.731515970032,-0.626052631824,1.0
0.14962052078,1.24000574432,-1.0
1.16720084422,0.521580749715,-1.0
-0.436063303539,0.043680311306,1.0
-0.827638902506,0.275166403707,1.0
1.36953107467,0.971233523422,-1.0
0.690612759144,-1.27804624607,-1.0
1.26986688391,0.575808793135,-1.0
0.208866020688,-0.146742455013,1.0
-0.437203222578,0.52116507147,1.0
-0.378363762158,-0.0769780148552,1.0
-0.423820115256,-0.836137209863,1.0
-0.560756181289,-0.41037775405,1.0
0.336052960763,-0.224802048045,1.0
-1.33543072512,-0.990358481473,-1.0
-0.0289733996866,0.441010128386,1.0
-1.3193906415,-0.37764709941,-1.0
-0.808411080806,1.2283790386,-1.0
1.35995943884,1.12161870845,-1.0
-0.872069364163,-0.252522725967,1.0
-1.88887517471,0.144098536459,-1.0
1.60845822722,-0.774759253864,-1.0
-0.358639909549,0.784305199745,1.0
0.520332593218,-0.62185400704,1.0
0.306204273961,0.25448089669,1.0
-1.51072939376,0.00594704976351,-1.0
0.956067338203,-0.533023015577,-1.0
0.288866739458,-0.725155662248,1.0
0.403468553933,-1.75945770781,-1.0
0.0859415686163,-0.958846823471,1.0
0.381957047469,0.0124143718471,1.0
0.336004016976,-0.259620737798,1.0
1.02869639688,-0.785051442286,-1.0
-0.181058441906,0.00266871780379,1.0
0.279139768315,0.148068778283,1.0
-0.700587484192,0.118422440942,1.0
-0.474343136475,-0.162548759675,1.0
-1.29581526521,0.755926314388,-1.0
0.140673267698,-1.60264376179,-1.0
0.328196143279,0.444738575921,1.0
-0.940761503292,-1.00437673463,-1.0
0.4177654822,1.11423358886,-1.0
-0.802874871784,-1.27790346857,-1.0
-0.596842011584,0.593623894204,1.0
-0.112331263254,0.174318514314,1.0
-1.45753325136,-1.30679050369,-1.0
1.63561447039,0.27394296313,-1.0
0.113120402388,0.0204651461722,1.0
0.753405102224,0.1938301221,1.0
-0.538129041247,-0.000723035827331,1.0

第 2 部分：分析学习曲线

对于该测验的第 2 部分，你可以查看你之前绘制的曲线并判断三个模型中哪个模型欠拟合，哪个过拟合，哪个正好。

欠拟合过拟合练习

QUIZ QUESTION: :

根据上述模型曲线，哪个模型欠拟合，哪个过拟合，哪个正好？

ANSWER CHOICES:

模型	过拟合，欠拟合还是正好？
欠拟合
正好
过拟合

SOLUTION:

模型	过拟合，欠拟合还是正好？
欠拟合
正好
过拟合